A Virtual Experiment Design Approach For Big Data Based On Containers And Python Language

 

Dazhou Li^1* and Wei Gao²

¹Dr. Dazhou Li, Shenyang University of Chemical Technology, China, lidazhou@syuct.edu.cn

²Prof. Dr. Wei Gao, Shenyang University of Chemical Technology, China, gaowei_syuct@foxmail.com

*Corresponding author

 

Abstract

Given the inconsistency between the experimental environment and the production environment, the high hardware cost of the big data production environment which is difficult for ordinary universities to bear, and the complicated installation of software related to the big data field which is not easy to reproduce and other practical teaching problems in the current stage of big data experimental teaching, the advantages and disadvantages of the existing solutions are analyzed and a virtualization method is proposed to virtualize hundreds of thousands of virtual The design method of Big Data virtual experiments based on containers and Python language is proposed to meet the practical teaching needs of undergraduate Big Data courses by virtualizing hundreds of virtual servers on several physical servers to construct a private cloud of Big Data experiment servers within the university. By optimizing the design for a small-scale server cluster environment in universities, the redundant modules are streamlined, and only teaching experiment-related modules are retained, geared towards teaching simulation, saving the limited funds of ordinary universities and revitalizing state-owned assets; at the same time, open-source software is used to avoid intellectual property costs in future teaching sessions.

Keywords: Container technology, Python language, big data technology, virtual simulation experiments.